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8.4 modeling misregistration and related effects 
ON multispectsal classification* 

Fred C* Billiogeley 

Jet Propulsion Lsborstory» Cslifornis Institute of Technology 
Pessdens* Cslifornis 91109 


INTEODUCTIOK 

Spectral analysis generally takes the form of multispectral classification 
in which the classification is done by comparing the sample measurement vector 
to the statistics of the set of known material vectors (training statistics) 
representing all possible classes, and by using one of several decision methods, 
determining which of the knowns it most nearly matches. 

The problem pursued will be the effects of misregistration on the accuracy 
of multispectral claasif ication in answer to the question: 

What are the effects on multispectral classification accuracy of 
relaxing the overall scene registration accuracy from 0.3 to 0.5 
pixel? 

The misregistration is but one of a group of parameters tnoise^ class 
separability, spatial transient response, field size) which must all be 
considered simultaneously • The thread of the argument (which will be 
discussed in detail below) is this: any noise in the measurements (due to the 

scene, sensor, or the analog to digital process) causes a finite fraction of 
measurements to fall outside of the classification limits* For field 
boundaries, where the misregistration effects are felt, the misregistration 
causes the border in a given (set of) band(s) to be closer than expected to a 
given pixel, so that the mixed materials in the pixels causes additional 
pixels to fall outside of the class limits* Considerations of the transient 
distance involved in the difference in brightness between adjacent fields, 
when scaled to "per pixel", allows the estimation of the width of the border 
zones* The entire problem is then scaled to field sizes to allow estimation 
of the global effects* 

This approach allows the estimation of the accuracy of multispectral 
classification which might be expected for field interiors, the useful number 
of quantization bits, and one set of criteria for an unbiased classifier* 


*This paper presents the results of one phase of research performed at the 
Jet Propulsion Laboratory, California Institute of Technology, sponsored by 
the National Aeronautics and Space Administration under Contract NAS;-100* 


410 


original 

Of POOR QUALITY 



CONCLUSIONS 


The following briefly etated conclusions are developed in detail in the 
body of the report* 

o The difference between 0«3-and 0*5-pixel misregistration 
is in the noise for multispectral classification* 

o Precision users may have to reregister image segments any* 
vay» making extreme registration precision by the system 
of less importance* 

o Interpolation algorithm choice is relatively unimportant « 
provided a higher*order interpolator is used* 

o If small fields are important • small pixels are more important 
than sensor noise contributions* 

In addition» several observations result: 

o System registration to 1*2 pixels should satisfy users 
of film products* 

o There is a grey area of 0.5 to 1*2 pixels in which the 
requirements for high precision are not well justified* 


THE BASIC MODEL 

The expected effect of misclassif ication may be estimated by a simple 
first*order approach, because the differences in classification accuracy 
between the many classification schemes and conditions that have been tested 
are overshadowed by the vagaries in the data and assumptions in the 
classif ication process, so that higher order analysis will contribute little 
additional understanding* 

Consider first the probability of correct identification of a field 

interior pixel. Field interiors are nonuniform because of the combined 
effects of sensor noise, scaled to equivalent reflectivity (NEAp ) and 

inherent nonuniformit ies in the field itself* The overall brightness 
distribution iz considered to be Gaussian * this is approximately true for 
field interiors, although the distribution deviates considerably toward 
bimodal for nixed materials at field borders* 

The combined . ffect of these various noise sources produces a finite 
probability of mi sclassif ication* (Figure S*l) The first*order estimate 

considers the totil variance caused by the scene, sensor and quantitation as 
compared to the defined class sire limits, however these are determined* 
Similar, but relatively second-order, effects ®ay be expected with a higher 
order analysis* t*oper classifier training, resulting in accurate limits, is 
essential (Hixson et al, 1980). 

For simplicity, and because of the later desire to misregister one (or 

more) of the bands, the discussion will assume that spectral bands as sensed 

will be used, and that for recognition, the unknown pixel must fall between 
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appropriate limita in every band ceated. Yherefore, brightneaa ouCaide of a 
limit in any one band ia aufficient for rejectiooi ao that ve need to consider 
only one band at a time* 

The probability of a aample being within the claas limits can be derived 
by aaauming that an cnaemble of clean aignala from a aeries of areas of the 
same material can be anywhere within the quantising range with uniform 
probability, but that individual aamples are perturbed by the Gaussian noise 
with a distribution equal to <r , The probability diatribution of the aignal 
plus noise is found by convolving the probability diatribution of the signal 
with that of the noise. The probability of correct class assignment (i.e., 
the pixel is within the class limits) is then found by integrating the 
probability distribution between appropriate class limits (Friedman 1963). 
The result of this calculation is shown in Figure S-2. In the useful range 
of (3 (3<P< 7), the curve can be approximated by 

plog P - - 0.40 

where P ■ probability of correct classification, and 
n class size 

P* , with class size end a in the same units. 

^ scene 


Sources of noise will be the scene itself and the sensor, both assumed to 
be random for this analysis* The root mean square (nns) sum is taken to give 
the total effective noise* A number of pixel measurements may be averaged 
together to reduce the noise before classification* This final noise figure 
■ay oe compared to the vidth of the class to give p , from which the 
probability P of correct classification may be estimated* This leads to the 
Classif ication Error Estimator, Fig* S-3* 

As an example, consider a scene having a f ield*interior variation of 32, 
to be viewed with a sensor having a total noise figure of IX* The total 
effective noise seen by the classifier (upper left) will be the rms sum of 
these, or 3*16X, which for a total 0-2S5 digital number (dn) range, would be 
8*1 dn. If the class width (determined by the classifier algorithm) is 23 dn 
(right center) the P* 3.1, giving P « 0.742 (right lower). If this P is 

not accurate enough for the analysis, several pixels must be averaged( right 
upper): a 2x2 averaging will raise p to 6*2, giving a new P * 0*86* 

Considering P in this way allows an estimation of the total noise 
permissible as it affects the attainable classif ication accuracy* If the 
amount of scene noise to be encountered in a given classification task can be 
estimated, the allowable extra noise from the sensor and quantization can be 
specified by estimating the loss of accuracy of the classif ication caused by 
quantization error* This leads to an estimate of the number of bits which 
will be useful* 


Define the perfect sensor as having no random noise nor quantization error 
ii.e*, an infinite number of bits)* This will define (for nxn pixels averaged) 


p. 


class size 


scene 


and p - lO'^'^Po 

o 
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For the reel sensor, P<Po beceuse of the finite t^gensor ^quentisetion* 

The new probability of correct classification P is related to by: 

P - Pp(p/Po) 

A plot of the loss in classification accuracy vs. P^ is given in Figure 

S-*4, for the parameter families P/l^o and sensor^ ^pcene* Hoise 

allocation starts with defining the desired P^ and ascertaining that the 
required ^ocan be obtained. Definition of the allowed AP determines (e.g., 
from the graph) the allowed ^gensor^ ^ scene* ^ estiaution of Che scene 
noise for which the other conditions apply allows Che calculation of Che to'al 
sensor noise allowed. The final step is to partition this noise between 
(tensor random noise and quantisation noise. 

For example, let the desired * 85)i and allow no more than 2X loss 

due to the total sensor noise. The no-sensor-noise Po must be s5.7 io give 

Pq. Then, from Figure S-4, the allowed ^•ensor * ® ^scene* 

the tcene has a ^ scene “ allowable ^sensor * 0.6x2Z * 

1.22, which must be partitioned between M£Ap and the qu antitation noise. 
For KE Ap • 12, the allowable O qxiant * \ 1.2"^ ~ 1^ ’ » 0.662, which 

can be met by 6-bit quantitation. 

TWo observations are important here: (1) Increasing the number of bits of 

quantitation produces improvements which asymptotically approach tero, as each 
successive bit reduces the step site by a factor of 1/2. (2) A scene having 

as little as 22 variation is a very uniform scene. Since this noise is rms*d 
with the sensor noise, it will overwhelm any but a very noisy sensor. 

Therefore, for purposes of multi^spectral classification, more than six bits 
would seem to be unnecessary. 


EDGE EFFECTS 

To this point, the analysis is based on pixels well inside uniform fields 
and well away from field boundaries. A number of experimenters have spent 
appreciable time discovering that classification accuracy falls oft at 
boundaries due to what has become known as the mixed-pixel effect. We will 
start at Chat point and attempt to model the effect to allow us to quantify 
our expectations. 

Ve assume as a starting point that all the spectral bands used in 
classif icat ion, whether obtained from one date or series of dates, are in 
perfect registration. This means that when the pixel grids from each band are 
aligned the data contents (field borders, roads, all features) are also 
aligned - note that this is more than simply having all internal disto tions 
removed, which is all that most geometric rectifications accomplish. 
Misregistration will (later) be considered as the lack of alignment of the 
pixel grids; because the computer can only work with pixel grids, aligning 
these pixel grids appears to the computer as a shift in the 

boundaries. We will assume that training samples are accurate and that class 
limits have been set from these by the classifier chosen* The classification 
is modelled as follows: signature shifting in any individual band will tend 
to cause misclassif ication, so that the situation may be treated one band at a 
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tine. The effects of pixel mixture in ell bends aey then be nos'd together if 
desired. The entire enelysis simplifies to the eonsidcretion of the trensient 
intensity snift serose field boundsries ss compered to the dess limits end 
the noise components of the messurement. 

The first step in snslyring the spstisl extent of pixel mixing ecross 
borders is to estimete the shepe end extent of the trensient intensity shift. 
If the impulse response functions or the moduletion trenefer functions (MTFs) 
of the verious components (end, hence, the entire system) ere known, e precise 
trensient response ewy be cslculeted. For exemple, the epee if icet ions for the 
Thesietic Mepper for Lendset D cell for s 22 to 982 time equivelent of ebout 2 
pixels implying e 102-902 trensient response of ebout 1.3 pixel. The 
precticel result of this is thet the "infinitely eherp" edges of the reel 
scene will be softened by the filtering effect of the scanning eperture 
(essumed to be rectenguler end heving uniform response) end it is this 
softened trensient response which is eempled. Interpoletion required for 
reiistretion will cause some further softening, end the use of any of the 
competent higher-order interpolation functions (sinx/x, TRW cubic convolution, 
modified cubic convolution, other splines) will have minor effects of the rise 
time. A total '^10-90 (transient response from 102 to 902) of 1.5 pixels 
with no ringing vi' 1 be used as e surrogate global value. 

The transient situation across a border is sketched in Fig. S-S. We are 
concerned here with the decrease in probability that a given pixel will have a 
value within the class limits as that pixel moves toward the boundary, as 
shown in Figure S-6. The analysis only needs to determine the area under the 
mormal curve (assuming the noise is Gaussian) between the limits as determined 
by the classification class site and the offset from the "field interior value" 
caused by the mixture. The important scaling involved is the amount of signal 
shift caused by the transient total shift T, as related to the desired class 
site S, for a given P . The left portion of Figure S-7 reflects this shift in 
brightness (vertical axis) as it effects the area within the class (the 
probability of recognition). 

The transient rise distance estimated for the Thematic Mapper has very 
close to a Gaussian shape and a *^10-90 * pixel. The amount of 
brightness shift is the differ ^nce between the brightness of the field under 
consideration and the adjacent field which is causing the shift. The 
important intensity relation is the magnitude of this shift, T, as related to 
the sixe S of the class being tested by the ratio T/S. These curves, for 
various T/S, are combined with the probability curves of the previous 
discussion in Figure S-7. From this may be estimated the loss in probability 
in classification of pixels near borders. 


BIAS IM FIELD SIZE ESTIMATION 

It can be appreciated that several things are happening simultaneously: 
If the lower limit of field B and the upper limit of field A have a gap 
between, pixels "lost" by field B will not be picked up by field A, and will 
be considered unknowns and not be counted in either field. The lost pixels 
will be some interior pixels, due to insufficient P , and a large number of 
near-border pixels, resulting in apparent field site loss. Only if the lower 
limit of field B and the upper limit of field A are coincident will 
pixels lost from one field be picked up by the other, and vice versa, to give 
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complete account of all pixela. For the field siae eatiaator to be unbiased , 
Che loae-and-pickup in both direcciona muct cancel; that ia, on the average 
the true border auat be located. The total effect vill depend on the ratio of 
Che number of border pixela to the number of field-interior pixela, and hence 
ia * furction of Che field ahape and aixe. 

Thia leada directly to Che required algorithm for field aixe eatimation: 
Firat divide Che acene ir to bloba, each of which ia aufficiently v .'orm, and 
with cloaed boundariea. Then for each blob (field) determine i... average 
brightneaa for all the interior pixela which are aafely away Ircm the border. 
For each aegment of the border, the correct field edge deciaion level ia 
midway (in a 'a) between Che average brightneaa of the two fielda on either 
aide. After the bordera are located uaing thia criterion, the field interiors 
may be reclassified using the classification limits as determined from the 
training samples* 


EFFECTS OF MISREGISTRATION 

In preparation for estimation of Che misregistration effects, an analysis 
will first be made of the expectations of registered data and the sensitivity 
Co the various parameters estimated. The starting model used ha.^ rectangular 
fields aligned with Che pixel grid. Pixela arc grouped into four tones: 1) 

Interior (i)-thoae with centers 2 or more pixels inside borders, 2) Inne 
border (ib)-pixels with centers 1-1/2 pixel inside borders, 3) Outer border 
(ob)-pixels with centers 1/2 pixel inside borders, 4) Exterior border 
(xb)-pixels outside the borders, with centers 1/2 pixel outside. Estimates of 
classification accuracy for each cone are obtained from Figure S-7. Tlie totn* 
estimate of classif icstior accuracy is the sum of pixels in ‘isch rone 
multiplied by the corresponding cone accuracy estimate. Late-, the field will 
be misregistered, changes in the number of pixels in each tone calculated, and 
Che probabilities again aummed. The following parameters are required: 

r - the field shape ratio, length of long side/length of short side 
T - trarsient brightness difference between field being considered 
and its neighbor 
S - decision class site 

T - transient distance for lOZ to 90Z response 
|3 - class sice S/a of Gaussian noise 

The following global values selected for the parameters are considered to be 
representative : 

r - 2 

T/S « I to 5 

T « 1.5 pixels 

p ■ 3 to 5 

After the parameters r, T/S, T , and (i are selected, the resultant (from 
Fig. S-7) probabilities are substituted for the brightnesses in the various 
cones to produce a "probability image" aligned with the desired output pixel 
grid. Tlie probability assigned to a pixel at a given location represents the 
probability that that pixel will have a brightness falling within Che 
classification limit determined by the classifier, for Che given spectral 
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band. The total probability of correct claaaification it given by 




Pi"' 


^ Pib"ib ^ Pob"ob ^ Pxb"xb j 


where n^ ia ch field width (abort aide) in pixcla, and 0 £« n^^t 

0^5 are the mubber of pixela in the varJoua sonea. Uaing theaa valuea, the 
global tatimats of the probability of correct claaaification with no 
■iaregiatration ia given Figure S-8 for three valuea of T/S. The predominant 
effect ia the pixel mixture (the effect of T/S). Aa expected, thia ia worst 
for small fields (n^ small) because of the larger v>rcentege of border 
pixels for these fields. Mote that for T/S ■ 1, decision level midway between 
brightnexset of adjacent fields, no probability loss occurs, even with small 
fields. Unfortunately, this desirable condition cannot be syatema'-ically 

obtained. 


M;<1R£GISTRAT10M of congruent fields 


The initial model for miare^istration is a displacement of d pixels, eq-al 
in both X and y. The result of this misregistration is that nome .re^ Is lost 
from the external border, causing a fv ther classification acccrncy decrease. 
The misregistration loss as seen by the external border loss is given by 




The basic character of this misregistration loss term is 1/n^, so that it 
will have a slope approximately equal Co -1 on a log-log p'ot vs n^. The 
precise results depend critrcally on the vilues of estimated for the 
Pjj^ from Figure S-7; 


T/S 

.8 

T- 1 T - 1.5 T - 2 


3 

.10 .14 .20 

1 

5 

.02 .025 .07 


7 

0 .01 .04 


3 

0 0 0 

2 

5 

7 

0 0 0 

n n n 


Using these values, the loss AP due to displacement niisregistiation is 
plotted in Figure S-9 for various parameter combinations. 


MISREGISTRATION DUE TO NON-CONGRUENCE 
1.) SIZE AND RATIO (ASPECT) CHANGES 

Size and aspect ratio changes can come about from several c ^ e: such as 

scan velocity or altitude changes, and if uncompensated can cause additional 
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■i«retitcr«tion errors* Progressive ■is'egistretion from a point of accurate 
registration will be caused by both causes (Figure S'lOa); the aodeling of 
this effect considers first that sixe changes M “ n'/n will cause a shift in 
points n to points n' both vertically and horiaontallyt and then that changes 
in aspect ratio will cause further shifts in the borisontal position of 
vertical borders by changing the field shape ratios by the factor R • r'/r* 
The resulting shifts are: 


An^ • (M - 1) n^ and An^ • (HR - 1) rUy 


For analysis, this shift will be divided around the borders syvetrically as 
optinuB field registration is accoatplished (Figure S*lOb). lira cases siust be 
distinguished (using scan velocity as a surrogate cause): 

Case 1: A slow sc^n decreases pixel spacing and puts aore pixels into a 
given field. When these are placed into the output grid, the field appears 
stretched. The field as defined by the other (correct) bands now covers only 
part of the stretched field, so that the classification tends to see only 
interior pixels, and the accuracy will increase, ultiaately reaching the 
f ield'interior accuracy. The sites of the border errors are: 

ej^ • *2 (H “ l) and e2 * "2 ^*^1 


Case II: A fast scan has the opposite effect, causing the field to appear 
smaller and the analysis pixels defined by the other bands now include more 
exterior pixels. The classification accuracy will decrease. 


For fast scan, the sitaller apparent field covers an area expressed as a 
fraction of the total* 


Fractional Areas: 


I 

I 




r„7 


xo 


2Nni 4 + 4 


(Interior) 


(External Border) 


The total expected probability is 


^tot “ ^iPi * Pxb * 

Since the external border pixels are now included within the analyzed 
field, but with a low probability, the fractional area RN^ represents 
approxioately the fraction of the basic field-interior accuracy be 

expected. Since the total size shrinkage ^ in pixels^) is sma for small n^, 
only larger n^ need be considered, a the 1/n^ term stay be dropped. 
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Ihi« allows Ftot to be approxiaatad for r * 2 by: 


Ptot ~ ♦ -—Pxb 

“l 

For large fields, Che probability is aeen to be independent of field aise, and 
only weakly dependent (because of low p^b) fox sull sixes. 

2.) VAVy BORDERS AND MULTIPLE ACQUISTIONS 

For single'band analysis, with borders distorted so that there are pixels 
both inside and outside of the analysed area, soaw pixels will have increased 
probabilities of correct classification and sone will have less. The decrease 
in probability across border is (very) approxiaately linear, so that the 
(signed) average displaceccnt will wodel the effect. 

For wulci-band analysis. Chose pixels having a low probability of 
classification will have the largest effect as the net probability at each 
pixel location is the product of the probabilities obtained for each 
acquisition (band). In this case the ms diaplaceaent will produce a better 
Bodel of the effects. 


SOME OBSERVATIONS 

I. ON BASIC CLASSIFICATION 

o The total noise figure (compered to the class site in a given 

dete.iDination) controls P , and in turn controls the iMximuai 
attainable classification accuracy. However, for practical range of 
V 7 , increasing P has only a aoderate effect. 

o Because of this, if small fields are most important, the reflected 

energy might be more profitably divided into smaller pixels, even 
at the expense of NE^^ . As this will cause an increase in data 
rate, optimum coding should be investigated. The possible noise 
introduced xn reconstructing the data will cause some further 
decrease in the overall effective ME^p and so decreases p . But 

since there is smaller sensitivity to p than to l/nj, there should 
be a net gain in utility. 

o Increasing the number ot bite of quantisation produces improvements 
which asymptotically approach zero, as each successive bit reduces 
the step size by a factor of 1/2. 

o A scene having as little as 22 variation ie a very uniform scene. 

Since this noise is rme'd with the sensor noise, :t will overwhelm 
any but a very noisy sensor. Therefore, for purposes of 
multi-spectral classification, an extreme number of bits would seem 
to be unnecessary. 

II. ON EDGE EFFECTS 

o For accurate field size estimation, the decision brightness mu«t be 

halfway between the brighnesses of the fields on cither aide of a 
given boundaiy. This means that classifiers sec for material 
identification will in general produce errors in field size. But the 
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ficld-iottrior brightness is incressingly herd to estimste lor sull 
fields becsuse of the fever interior pixels, 
o It is ioportsnt to keep the trensient response distnnce end the 
sccottpsnying snaple spacing soall» to get as many pixels into a given 
ground distance as possiule* Field area errors becose large at 
* 3 or less* the transient distance vust also be matched between 
spectral baids. 


111. ON HISREGISTBATIOM 

o For large T/S (i.e., 2 or «ore) the edge effects are so great that 
the base probability is drastically affected » and the external border 
pixels have aero probability of being within the class limits* For 
this reason, there is no misregistration effect for large T/S* 

o Square fields show the siost misregistration loss, when scaled to 

o A shape ratio r*2 is believed to be representative* 

o Hisregistration loss decreases with higher p • However, these losses 

in general are sms'l to begin with, and the discussion calling for 
sacrifice of i5 to gain smaller IFCV (more pixels ni into a given 
field) would seem to override* 

o Increase in T decreases the basic accuracy of edge pixels and also 

increases the misregistration losses. 

o Geometric rectification and registration procedures must not only 

xemove the internal distortions but must also produce pixels on a 
defined (preferably ground-referenced) grid. Current procedures do 
not do this. Without this reference grid, users will have to 

re-interpolate before multi^temporal data can be compared. 

o Scale and aspect ratio errors will have only minor effects on 

mcderate-area problems, but they will cause problems in correlating 
over large distances. 

o Altitude relief displacement will require users to use many control 

points to register images in areas of high relief. 

o Unless standard reference grids are established, users requiring 

registration will have to interpolate every image, even in low relief 
areas* 

o For single-band analysis, the algebraic average of the displacement 

may be used. For multi-band analysis, with erratic errors in 

location among the bands, the lowest probability of correct 
classification holds and the ims of the displacements is appropriate. 
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AN UNANSWERED QUESTION 

This report aodclt the potential ■iaregiatration effecta on aultiapcctral 
classification accuracy* It oay allow the coopariaon of the werioua tests and 
aiwu]?>tions, and pointa out the variablea which auat be reported for those 
eiaulations to allow their walidation* It doea not answer the following 
question; Given a certain loss in accuracy due to nisregiatration, how does 
that daaage the ability to use the data analysis results? These evaluations 
will be discipline dependent, and mat be sought separately. 
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AREA IS PROI>ORTIONAL TO 
PROBABILITY OP BEING 
IMTHIN THE 0>SS LIMITS 


Figure S-1 Effect of Molac on the FrcAablllcy of Correct Multl> 
•pectrel Cleeelficatlon 



P = CLASS SIZBrms NOISE 


Figure Given a Signal Uniforaly Frobablc over the Dynaaic Eange, and 

Gauasian Noise vith standard Deviation ■ <7 . The curve shows 
the Probability of Correctly Necognixing a class corresponding 
to the Noiseofree Signal as a Function of the Ratio 
^ ■ class site t a • 
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Figure S>4 Loss in Clessiflcstlon Accurecy due to Noise 
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PIXEL SPAQNQ 


Figure S-5 Cross Csction of brightness trees ecross s boundezy between 
two fields « showing the distence required for the bright* 
sees trensltion. 



Figure S-6 Ihe distribution of "field" pixels moves down the txensition 
curve SB the measurement point moves toward the boundary. 

The shaded area is the proportion trhich will be correctly 
classified. 
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nxa aNTCR distance from soroer - fuels 

Figure S>7 Conbined Curves for Translation of Pixel Distance from 

Border to Shift in the Brightness (Scaled in Dnlts of Class 
Sits, S ) to Probability of Correct Classification. 
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Figure SS Probablllcy of CorrecC Classification using Global 
Parameters, for Perfectly Registered Pixels. 

One Spectral Band Only. 




REGISTRATION POINT 
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o. GENERAL CASE 


b. BEST REGISTRATION CASE 

(NO TRANSLATION OISPLACEV.ENT) 


Figure S>10 Construction for Estimating Misregistration Caused 
By Sire and Aspect Errors. 
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Figure S-9 Loss of Classification 
‘Accuracy due to Misregistration of 
One Band, for Various Parameter Com- 
binations. 

Class slte/(T of noise 
r • Field Shape Ratio, long/short sides 
T ■ 10-90X transient distance 
Oj “ length of short side, pixels 
d • displacement, pixels 
A? • loss in probability 
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